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AIR  FORCE  SPACE  COMMAND  ACTION  OFFICER  GUIDE  TO 
RELIABILITY  AND  MAINTAINABILITY 

PREFACE 


The  Secretary  and  Chief  of  Staff  of  the  Air  Force  stated 
the  Air  Force  commitment  to  reliability  and  maintainability 
(R&M)  in  their  joint  memorandum  of  September  1984  and  renewed 
the  commitment  in  July  1986.  An  Air  Force  R&M  2000  Action 
Plan  was  developed  to  provide  general  policy  and  guidance  to 
institutionalize  R&M  in  the  way  we  do  business  -  both  now  and 
in  the  future. 

In  response,  Air  Force  Space  Command/LKYY  outlined 
Command  goals  in  the  Reliability  and  Maintainability  Program 
Plan  which  provides  approaches  to  improve  R&M  performance  and 
achieve  AF  R&M  2000  goals.  This  R&M  Action  Officer  Guide  is 
an  integral  part  of  LKYY's  efforts  to  expand  the  R&M  Program 
Plan  so  that  AF  Space  Command  action  officers  can  understand 
R&M  parameters  and  how  they  impact  system  capability  and 
performance.  This  guide  illustrates  how  R&M  principles 
should  be  applied  to  new  acquisition  programs  as  well  as  to 
fielded  systems. 

I  fully  endorse  this  Guide  and  expect  all  AF  Space 
Command  action  officers  who  are  involved  in  the  acquisition 
process  to  not  only  familiarize  themselves  with  its  contents 
but,  more  importantly,  to  utilize  this  information  to  ensure 
we  get  low  cost,  high  performance  space  systems. 


DONALD  J.  KUTYNA 
Lieutenant  General,  USAF 
Commander 
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1.0  INTRODUCTION 
1.1  BACKGROUND 


Although  R&M  had  long  been  recognized  as  important 
considerations  in  fielding  major  weapon  systems,  the  emphasis  on 
R&M  in  an  acquisition  program  usually  took  a  back  seat  to  cost, 
schedule  and  performance,  especially  when  trade-offs  were 
required  to  offset  budgetary  constraints.  These  cost  cutting 
practices  reduced  up  front  development/procurement  costs  but 
resulted  in  increased  long  term  operations  and  support  costs.  To 
combat  the  mounting  cost  of  supporting  our  space  and  ground 
systems,  the  Secretary  and  Chief  of  Staff  of  the  Air  Force  have 
promulgated  vigorous  Air  Force  commitment  to  R&M: 


"For  too  long,  the  reliability  and  maintainability  of  our 
weapon  systems  have  been  secondary  considerations  in  the 
acquisition  process.  It  is  time  to  change  this  practice  and 
make  R&M  primary  considerations." 

"We  must  emphasize  R&M  throughout  the  acquisition  process— 
from  requirements  definition,  throughout  concept  development, 
design,  production  and  acceptance.  Everyone  must  ensure  R&M 
requirements  are  met  through  every  step  of  the  process.  R&M 
must  be  coequal  with  cost,  schedule  and  performance  as  we 
bring  a  system  into  the  Air  Force  inventory." 

(General  Charles  Gabriel,  Chief  of  Staff,  USAF, 
and  Mr.  Verne  Orr,  SAF,  17  Sep  1984) 


1 . 2  PURPOSE  OF  GUIDE 


This  guide  is  designed  to  help  action  officers  responsible 
for  managing  acquisition  programs  better  understand  the 
interrelationship  between  R&M  performance  and  operational 
capability.  It  won't  turn  you  into  an  R&M  engineer. 

This  guide  is  designed  to  be  used  with  the  Command 
Management  Guide  developed  by  XPT  and  the  Reliability  and 
Maintainability  Terms  for  Space.  Space  Surveillance,  and  Missile 
Warning  Systems  technical  report  published  by  LKY. 

The  XPT  guide  is  a  comprehensive  document  that  provides  a 
single  source  of  reference  for  the  staff  officer.  It  contains  an 
overview  of  the  acquisition  process  and  outlines  the  command 
manager's  responsibilities.  For  more  information,  contact  HQ 
AFSPACECOM/XPT,  Peterson  AFB,  CO  80914-5001  or  call  AV  692-5668. 
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The  LKY  technical  report  details  both  mission  and  logistics 
Support  R&M  parameters .  The  terms  and  definitions  contained  in 
the  report  are  approved  for  use  in  SOtfS  and  SORDS .  To  obtain  a 
copy  of  this  report,  contact  HQ  AFS PACE COM/ LKY Y ,  Peterson  AFB,  CO 
80914-5661  or  Call  AV  692-3286/5898. 


1.3  OVERVIEW 

The  basic  guide  discusses  the  role  of  R&M  in  requirements 
determination  and  system  acquisition  process.  Appendix  A  is  a 
primer  of  basic  R&M  concepts  and  computations.  Appendix  B  is  a 
checklist  designed  to  evaluate  R&M  content  in  System  Operational 
Requirements  Documents  (SORDs) .  Contract  data  items  applicable 
to  R&M  are  listed  in  Appendix  G. 


2.0  R&M  IN  THE  SYSTEM'S  LIFE  CYCLE 


2.1  OVERVIEW  OF  PHASES 

System  acquisition  programs  normally  go  through  several 
phases,  each  with  particular  milestones,  as  they  progress  from 
the  identification  of  a  need  to  final  operational  deployment. 
Figure  2-1  shows  the  acquisition  process  and  the  major 
events/documents  that  usually  occur  in  each  phase.  Phases  may  be 
combined  or  omitted  depending  on  circumstances;  however,  each 
phase  is  designed  to  develop  a  level  of  confidence  in  the 
solutions  offered  and  to  reduce  the  risks  of  entering  the  next 
phase.  R&M  is  a  key  factor  in  the  decision  process  as  the 
program  progresses  through  each  phase. 


2.2  MISSION  AREA  ANALYSIS  PHASE 

The  acquisition  process  begins  with  a  mission  area  analysis 
identifying  mission  requirements  and  assessing  the  Command's 
capability  to  fulfill  each  requirement.  Statements  of 
Operational  Need  (SONs)  define  new  requirements  that  cannot  be 
met  through  changes  in  tactics,  strategy,  doctrine,  or  training. 
The  SON  also  offers  a  possible  solution  involving  either  new 
equipment  or  upgrades  to  an  existing  system.  Section  3  describes 
placing  R&M  requirements  on  SONs.  Once  the  SON  is  validated  by 
the  Air  Staff,  it  forms  the  basis  for  writing  the  Program 
Management  Directive  (PMD) .  The  System  Program  Office  (SPO) 
selected  for  the  project  then  develops  a  Program  Management  Plan 
(PMP)  that  describes  the  tasks  necessary  to  develop  a  system  that 
fulfills  PMD  requirements.  The  operational  command  is  also 
tasked  to  develop  a  System  Operations  Requirements  Document 
(SORD)  which  amplifies  and  refines  the  operational  and  support 
needs  specified  in  the  SON.  R&M  considerations  for  these 
documents  are  discussed  in  Section  3. 


2.3  CONCEPT  EXPLORATION/DEFINITION  fCE/D^  PHASE 

In  the  CE/D  phase,  the  implementing  command  (normally  AFSC 
for  a  major  new  weapon  system)  may  have  several  companies  in 
competition  for  the  award  winning  design.  Operations  and  support 
planning  are  integral  activities  during  this  phase.  Contractual 
R&M  requirements  are  developed  to  the  same  extent  as  are  other 
performance  parameters.  As  system  operating  and  support  concepts 
are  further  developed  during  the  acquisition  process,  operational 
R&M  needs  and  the  corresponding  contractual  R&M  requirements  are 
challenged  and  refined.  The  objective  of  this  refinement  process 
is  to  have  R&M  needs  and  contractual  requirements  that  are 
validated,  consistent,  achievable  and  acceptable  by  the 
operating,  implementing,  and  supporting  commands  prior  to 
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developing  the  Full  Scale  Development  Request  for  Proposal  (FSD 
RFP) .  Section  5  explains  R&M  tasks  in  the  contractual  process. 

By  the  end  of  the  CE/D  phase,  decisions  affecting  70%  of  the 
Life  Cycle  Costs  (LCC)  have  already  been  made,  and  decisions 
affecting  85%  of  the  LCC  are  made  prior  to  Full  Scale  Development 
(FSD),  as  shown  in  Figure  2-2.  These  decisions  affect  system 
supportability,  R&M  characteristics,  and  LCC.  For  maximum 
benefit,  R&M  requirements  must  be  clearly  stated  in  early 
acquisition  documents  including  the  System  Operational 
Requirements  Document  (SORD) ,  the  R&M  Management  Plan  (RMMP) ,  and 
the  Test  and  Evaluation  Master  Plan  (TEMP) .  These  documents  are 
discussed  in  Sections  3  and  6. 


LIFE-CYCLE  COSTING  IN  SYSTEM  ACQUISITION 


SYSTEM  LIFE  CYCLE 


YEARS 


Figure  2-2 


CONCEPT  DEMONSTRATION/VALIDATION  (CD/Vi  PHASE 


When  the  exploration  of  alternative  system  concepts  has  been 
completed  to  the  point  where  the  selected  alternative  warrants 
system  demonstration,  the  program  enters  the  CD/V  Phase.  The 
purpose  of  this  phase  is  to  reduce  technical  risk  and  economic 
uncertainty  through  a  more  detailed  definition  of  the  system. 

The  System  Program  Office  (SPO)  works  closely  with  the 
contractor (s)  to  further  define  the  system  and  will  frequently 
task  them  to  build  prototypes  and/or  accomplish  desktop  analysis 
to  provide  necessary  detail.  These  details  are  included  as 
updates  to  the  documents  first  published  in  the  Concept 
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Exploration/Definition  phase.  Therefore,  much  of  the  R&M  work  in 
this  phase  of  the  acquisition  is  a  continuance  of  work  done 
previously.  R&M  requirements  are  allocated  down  to  the 
subsystems  and/or  equipment  items  within  the  system.  Certain 
meetings  (i.e..  System  Requirements  Review  (SRR)  and  System 
Design  Review  (SDR) )  are  held  to  review  the  system  requirements 
and  detailed  design.  R&M  requirements  for  these  reviews  are 
described  in  Section  5. 


2.5  FULL  SCALE  DEVELOPMENT  (FSD^  PHASE 

Upon  final  selection  of  a  design  concept,  the  system  is 
ready  to  enter  the  FSD  Phase.  The  implementing  command  will 
reaffirm  the  operational  need  for  the  system,  adequacy  of 
evaluation  of  the  alternative  approaches,  adequacy  of  the  test 
and  evaluation  approach,  sufficiency  of  cost  and  schedule 
estimates,  and  consistency  of  the  acquisition  strategy  and 
contractual  plan  with  program  characteristics,  requirements  and 
risk.  Once  this  review,  called  the  Secretary's  Program  Review 
(SPR)  is  completed,  the  implementing  command  will  enter  FSD. 
Contract  requirements  for  R&M  are  the  same  as  those  in  the 
earlier  phases  (see  Section  4) .  Once  the  contract  is  awarded, 
two  major  reviews,  the  Preliminary  Design  Review  (PDR)  and 
Critical  Design  Review  (CDR) ,  are  conducted  to  check  the 
contractor's  progress  and  ensure  the  technical  adequacy  of  the 
design  approach.  During  this  phase,  the  action  officer  must 
review  the  test  planning  documentation  to  evaluate  the  DT&E  and 
IOT&E  efforts  against  the  operational  requirements.  Once  the 
test  has  been  conducted,  there  are  reviews  of  the  test  results  to 
determine  how  well  the  actual  system  meets  operational  and 
support  requirements.  R&M  is  a  determining  factor  in  how  well 
the  system  passed  OT&E,  as  described  in  Section  7.  Finally, 
there  is  another  SPR  at  the  end  of  FSD  to  make  the  final  decision 
to  accept  the  prototype  as  a  one  of  a  kind  or  to  mass  produce  the 
system.  R&M  issues  for  this  SPR  are  identical  to  those  of  the 
previous  SPR,  described  in  Section  6. 


2.6  PRODUCTION/DEPLOYMENT  (P/Dl  PHASE 

Once  the  system  design  has  been  fully  developed,  approval 
can  be  given  to  enter  the  P/D  phase.  For  a  major  program 
involving  large  production  quantities,  producibility  and 
reliability  growth  are  examined.  Large  production  runs  don't 
normally  occur  for  the  typical  Space  Command  acquisition;  the 
prototype  system  frequently  becomes  the  operational  system. 
Sometimes  an  evolutionary  acquisition  strategy  is  used.  Under 
this  strategy,  system  capabilities  are  acquired  in  blocks.  Each 
block  is  sequentially  acquired  and  employed  and  becomes  the  base 
for  the  next  block.  This  strategy  is  used  primarily  on  systems 
which  push  the  state  of  the  art  or  on  which  future  requirements 
cannot  be  accurately  forecast.  Tracking  R&M  requirements  across 
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each  evolutionary  acquisition  block  is  critical  to  achieving 
operational  capability. 


2.7  OPERATIONS  SUPPORT  PHASE 

The  OS  phase  begins  with  the  first  operational  deployment. 
The  system's  operational  performance  is  tracked  and  problems 
identified.  R&M  parameters  affecting  mission  performance  or 
supportability  are  closely  monitored.  Equipment  replacement  or 
modification  may  be  required  to  maintain  system  level  R&M 
performance.  The  importance  of  maintenance  data  collection 
during  this  phase  is  addressed  in  section  9 . 


2.7.1  Initial  Operational  Capability  (IOC) .  At  the  beginning  of 
IOC,  the  using  command, (in  this  case  AFSPACECOM) ,  takes 
operational  responsibility  IAW  AFR  800-19.  Normally  some  time 
later,  management  responsibility  for  the  system  transfers  to  AFLC 
as  part  of  Program  Management  Responsibility  Transfer  (PMRT) ,  IAW 
AFR  800-4.  R&M  tasks  in  this  phase  are  normally  limited  to 
Follow  on  OT&E  (FOT&E)  (described  in  Section  8),  and  the 
establishment  of  a  Maintenance  Data  Collection  (MDC)  System  to 
track  actual  R&M  performance. 


2.7.2  Full  Operational  Capability  fFOCl .  Once  the  system  is 
fielded,  efforts  are  made  to  improve  system  effectiveness  and 
safety.  The  acquiring  agency  continues  to  correct  operational 
R&M  deficiencies  caused  by  material,  software,  or  firmware  design 
and  quality.  The  program  manager  corrects  operational  R&M 
deficiencies  within  his/her  responsibility  and  assures  that 
operational  R&M  needs  are  met.  The  operating  and  support 
commands  correct  deficiencies  that  are  the  result  of  inadequate 
operating  and  support  concepts.  After  PMRT,  the  operational  R&M 
performance  is  monitored  and  reported  through  a  MDC  system. 
Analysis  is  performed  to  assess  operational  R&M  performance, 
identify  deficiencies,  and  help  identify  corrective  actions. 

This  becomes  the  basis  for  a  new  SON,  thus  reinitiating  the 
process. 
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3.0  R&M  IN  PROGRAM  DOCUMENTS 

The  Statement  of  Need  (SON)  documents  the  operational 
command's  requirements.  Once  the  SON  has  been  validated  and 
funding  acquired  for  the  program,  the  program  enters  the  concept 
exploration  phase.  At  this  point  in  the  acquisition,  three 
documents  are  important:  the  PMD,  PMP  and  SORD. 

3.1  STATEMENT  OF  OPERATIONAL  NEED  fSON) 

A  SON  defines  an  operational  need  and  documents  official 
validation  of  the  need.  A  SON  is  an  AF-related  document,  other 
documents  that  describe  a  need  are  the  Joint  Statement  of 
Requirements  (JSOR) ,  Mission  Need  Statement,  Required  Operational 
Capability  (ROC)  or  Communications-Computer  System  Requirements 
Document  (CSRD) .  While  the  documents  differ  in  form  and  content, 
putting  R&M  requirements  into  each  of  them  follows  a  similar 
logic  flow. 

3.1.1  Defining  Requirements.  The  first  step  in  adequately 
preparing  a  SON  with  the  proper  R&M  terms  is  to  understand  that 
SONs  document  the  need  for  a  new  or  improved  capability,  identify 
operational  deficiencies,  and  define  constraints  on  acceptable 
solutions.  During  the  SON  development,  R&M  parameters  are 
defined  at  the  system  level.  Stating  top  level  R&M  needs  early 
in  the  acquisition  cycle  helps  determine  the  best  design  solution 
by  ensuring  R&M  considerations  are  an  inherent  part  of  the  system 
design  process. 

Inherent  in  the  mission  need  are  certain  top  level 
readiness  requirements.  These  readiness  requirements  should 
relate  to  the  peacetime  and  wartime  scenarios  envisioned.  For 
example,  if  you  are  writing  a  SON  for  a  ground  communications 
system,  general  wartime  requirements  from  the  War  and 
Mobilization  Plan-3  (WMP-3)  might  specify  the  equipment  be 
operated  24  hours  a  day/7  days  a  week.  If  the  equipment  is  on  a 
transportable  platform,  it  may  be  required  to  operate  without 
resupply  for  a  specified  minimum  period  of  time. 

3.1.2  Requirements  Allocations.  Once  top-level  operational 
requirements  are  established,  the  R&M  values  should  flow  down 
from  and  support  these  requirements.  Most  Space  Command  systems 
are  low-density  (normally  one  per  geographic  location  and  less 
than  20  locations  worldwide)  or  one-of-a-kind  space  and  attack 
warning  systems.  These  systems  often  have  to  meet  extremely  high 
R&M  effectiveness  criteria.  To  develop  or  evaluate  R&M 
requirements,  the  operational  need  must  be  completely  understood. 
Factors  including  the  system's  mission,  its  operations  concept 
and  maintenance  concept  must  be  considered. 
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As  a  minimum,  the  following  R&M  parameters  must  be 
determined  and  included  in  Section  IV  A  of  the  SON: 

—  Mission  Reliability 
—  Mean  Mission  Duration 
—  Mean  or  Maximum  Restoral  Time 
—  Availability/Dependability 

For  further  information  on  the  use/application  of  these 
terms,  consult  AFS PACE COM/ LKY  Technical  Report  88-1,  standard 
R&M  Terms  for  Space.  Space  Surveillance,  and  Missile  Warning 
Systems,  dated  20  April  1988. 


3.1.3  Quantifying  Requirements  Once  the  appropriate  terms  have 
been  selected,  the  next  decision  is  how  much  R&M  to  ask  for. 

This  decision  should  be  based  on  mission  requirements, 
technology,  and  comparable  current  systems.  This  task  can  be 
difficult  due  to  system  complexity  and  incomplete  historical  data 
on  predecessor  system  outages.  Also,  it's  not  always  possible  to 
find  a  "like  system"  to  use  as  a  basis  for  comparison.  Many  of 
our  new  systems  provide  capabilities  not  previously  available. 

All  possible  sources  of  information  should  be  tapped. 
Dialogue  with  contractors  and  engineers  at  Space  Division  and 
Electronic  Systems  Division,  as  well  as  the  AF  Acquisition 
Logistics  Center  (AFALC)  and  the  AF  Coordinating  Office  on 
Logistics  Research  (AFCOLR)  can  provide  valuable  information. 
Additionally,  AFSPACECOM/LKYY  personnel  will  assist  you  in  this 
effort. 

In  AF  Space  Command,  all  SONS  must  process  through  LKYY  to 
ensure  adequate  R&M  parameters  are  inserted  in  the  document. 
Figure  3-1  shows  the  AFSPACECOM  SON  validation  process  which 
ensures  R&M  is  adequately  addressed. 

3.2  PROGRAM  MANAGEMENT  DIRECTIVE  (PMD^ 

The  PMD  is  the  master  program  document  for  any  800-series 
acquisition.  It  defines  the  responsibilities  and  funding 
profile  for  the  program  and  is  the  key  decision-making  tool  to 
coordinate  the  efforts  of  the  using/developing/supporting 
commands.  As  far  as  R&M  are  concerned,  the  PMD  should  contain  a 
section  under  "Implementing  Command  Responsibilities"  that  shows 
what  tasks  are  going  to  be  performed  as  part  of  the  program. 

Those  tasks  include  conducting  an  R&M  program  per  AFR  800-18, 
publishing  an  R&M  Management  Plan  (see  Section  7.4  for  details) 
and  integrating  R&M  into  the  logistics  program.  R&M  tasks  must 
be  inserted  in  the  first  version  of  the  PMD.  Planning  up  front 
for  an  R&M  program  is  significantly  easier  than  trying  to  acquire 
it  later  because  the  common  "bring  money"  arguments  with  the  SPO 
can  occur  if  the  requirement  for  a  sound  R&M  program  is  not 
specified  up  front  in  the  PMD. 
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3.3  PROGRAM  MANAGEMENT  PLAN  (PMP) 

The  PMP  is  the  SPO's  response  to  the  PMD.  Instructions  on 
how  to  prepare  it  are  contained  in  AFR  800-2.  All  tasks  required 
by  the  PMD  should  show  up  consistently  in  the  PMP.  R&M  should, 
as  a  minimum,  be  included  in  section  4  (System  Engineering  and 
Configuration  Management) ,  section  5  (Test  and  Evaluation) ,  and 
section  9  (Logistics).  If  the  PMD  has  clearly  defined  the  R&M 
program's  requirements,  then  the  SPO  should  show  how  it  will 
complete  all  required  actions.  Inadequate  discussion  of  the 
R&M  program  in  the  PMP  is  one  of  the  first  indications  of  future 
trouble  in  getting  the  SPO  to  sign  up  to  R&M. 


3.4  SYSTEM  OPERATIONAL  REQUIREMENTS  DOCUMENT  (SORD) 

A  SORD  is  submitted  by  the  operating  command  for  each 
funded  program  as  tasked  in  the  PMD.  The  SORD  is  the 
requirements  and  planning  document  prepared  to  address 
operational  and  support  needs.  It  amplifies  and  refines  the  SON. 
AFR  57-1  specifies  the  SORD  format  and  directs  that  the  SORD  will 
quantify  the  following  R&M  parameters: 

A)  mission  reliability  and  maintainability 

B)  logistics  reliability  and  maintainability 
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C)  operational  availability  and  dependability 

These  values  are  described  in  Section  IV. A. 1  of  the  SORD. 
Section  IV.A.l.a  specifies  the  required  system  performance 
parameters  such  as  capacity,  mission  duration,  reaction  time, 
etc.  Section  IV.A.l.b  provides  the  mission  reliability 
and  maintainability  parameters  that  the  system  must  exhibit  (see 
the  R&M  Primer  and  AFSPACECOM  Technical  Report  for  Standard  R&M 
Terminology  for  further  details).  Section  IV.A.l.c  covers  the 
logistics  R&M  (expressed  as  Mean  Time  Between  Maintenance  (MTBM) 
and  Mean  Time  to  Repair  (MTTR) )  for  the  system.  Finally,  Section 
IV.A.l.d  describes  the  readiness  measures  in  terms  of  Ao  or 
operational  dependability  (Do) .  These  four  sets  of  parameters 
should  be  interrelated;  i.e.,  the  operational  parameters  should 
form  the  basis  for  the  R&M  parameters  that  follow.  AFSPACECOM 
SORDs  must  be  coordinated  with  LKYY  who  uses  the  SORD  checklist 
in  Appendix  B  to  assess  whether  R&M  issues  have  been  adequately 
addressed . 
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4.0  R&M  IN  THE  CONTRACTUAL  PROCESS 

The  SON  and  SORD  formalize  Space  Command's  operational 
requirements.  In  these  documents,  needed  capabilities  are 
described  in  terms  of  mission  requirements,  operational 
objectives,  employment,  and  support  and  maintenance  concepts. 

The  SPO  then  takes  the  operational  R&M  parameters  stated  in  the 
SON  and  SORD  and  translates  them  into  contractual  teotis  such  as 
Mean  Time  Between  Failures  (MTBF) ,  Mean  Time  to  Repair  (MTTR) , 
etc.,  to  meet  the  SON/SORD  requirements.  Next,  the  SPO  solicits 
private  industry  and  public  agencies  for  proposed  solutions  to 
this  need  in  the  Request  for  Proposal  (RFP) .  The  RFP  includes  a 
model  contract  with  a  SOW,  System  Specifications,  and  Contractor 
Data  Requirement  List  (CDRL) .  The  AFSPACECOM  action  officer  must 
ensure  that  contractual  terms  in  the  specification  and  R&M  tasks 
outlined  in  the  SOW  will  meet  SON  requirements.  The  R&M 
parameters  established  in  the  SON/SORD  and  translated  into 
contractual  terms  by  the  SPO  hold  the  contractor  liable. 


4 . 1  SPECIFICATIONS 

R&M  values  are  normally  contained  in  Sections  3.2.3  through 
3.2.5  of  the  specification.  Section  3.2.3  should  have 
reliability,  which  will  give  the  required  MTBF  or  R(t)  and 
mission  duration.  Both  MTBF  and  R(t)  should  be  specified. 

Section  3.2.4  gives  the  maintainability  specifications,  usually 
in  terms  of  MTTR.  Sometimes  the  SPO  uses  the  phrase:  "consistent 
with  the  reliability  and  availability  values,"  but  it  is  better 
to  specify  the  requirement.  As  shown  in  the  R&M  Primer  (Appendix 
A) ,  MTTR  is  a  contractual  term  that  does  not  include  logistics  or 
administrative  delay  times.  Therefore,  the  MTTR  value  must  be 
less  than  the  operational  value  given  in  the  SORD.  Section  3.2.5 
lists  the  availability  requirement  which  should  be  consistent 
with  the  R&M  specifications  above.  The  availability  formula 
given  in  LKY/TR88-1  should  balance  when  combining  the  RM&A  values 
in  these  three  sections. 


4.2  STATEMENTS  OF  WORK  (SOWsl 

A  SOW  is  the  part  of  a  contract  detailing  tasks  the 
contractor  must  perform.  R&M  tasks  in  the  normal  acquisition 
contract  include  conducting  an  R&M  program  IAW  MIL-STDs  471  and 
781.  References  are  also  included  for  the  development  of  various 
R&M  related  data.  The  details  of  the  data  requirements  are 
included  on  the  Contract  Data  Requirements  List  (CDRL) . 


4.3  CONTRACT  DATA  REQUIREMENTS  LIST  fCDRL^ 

The  CDRL  contains  the  requirements  for  data  to  be  delivered 
to  the  government  under  the  terms  of  the  contract.  The  CDRL 
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contains  or  references  the  required  format,  delivery 
instructions,  number  and  types  of  copies  required,  approval 
codes,  frequency  of  updates  and  required  delivery  dates.  The 
CDRL  references  a  Data  Item  Description  (DID)  which  describes  a 
standard  format  for  commonly  requested  data.  These  DIDs  can  be 
tailored  to  meet  program  specific  requirements  on  the  CDRL. 
Appendix  C  contains  commonly  used  R&M  DIDs.  AFSPACECOM/LKYY  will 
assist  the  action  officer  in  selecting  and  tailoring  these  DIDs 
and  in  preparing  the  required  CDRL  information. 


4.4  SOURCE  SELECTION  EVALUATION 

Source  selection  is  the  process  of  choosing  the 
contractor (s)  who  will  design  or  build  the  system.  Evaluation  of 
the  contractor's  approach  to  R&M  is  a  critical  activity  during 
source  selection.  The  contractor  must  show  his  or  her 
understanding  of  the  R&M  tasks,  ability  to  perform  those  tasks; 
and  a  sound  plan  to  integrate  the  R&M  tasks  into  the  entire 
system  engineering  and  logistics  support  effort.  Ideally,  a 
logistician  who  can  analyze  both  the  logistics  and  R&M  parts  of 
the  contractor's  proposals  will  be  a  member  of  the  Source 
Selection  Team. 


5.0  R&M  IN  DESIGN  REVIEWS 


5 .1  SYSTEM  REQUIREMENTS  REVIEW  CSRR) 

The  SRR  is  conducted  to  evaluate  the  contractor's 
responsiveness  to  the  SOW  and  to  ensure  the  contractor's 
interpretation  of  the  system  requirements  is  correct.  At  this 
time,  it  is  the  Action  Officer's  responsibility  to  ensure  the  R&M 
values  originally  placed  in  the  SON/SORD  were  accurately 
translated  by  the  contractor  into  the  Proposal  and  that  all 
elements  of  the  R&M  program  are  in  place.  This  task  should 
entail  matching  the  Proposal  to  the  SOW. 


5.2  SYSTEM  DESIGN  REVIEW  (SDR) 

At  the  SDR,  the  contractor's  proposed  approach  to  meeting 
system  operational  requirements  is  reviewed.  Normally  this 
approach  is  documented  in  a  draft  "A”  level  specification.  The 
R&M  emphasis  at  SDR  is  the  proper  and  consistent  establishment  of 
system  level  R&M  design  requirements.  The  R&M  requirements 
documented  in  the  "A"  level  specifications  are  the  inherent 
capabilities  the  system  must  possess,  given  a  stated  system 
operations  and  maintenance  concept.  "A”  level  specification 
requirements  are  derived  from  the  SON/SORD  requirements  by  taking 
into  account  the  impact  of  factors  external  to  the  system 
hardware  and  software.  Often  the  inherent  design  requirements 
(i.e. ,  "A"  specifications)  are  higher  than  the  SON/SORD 
requirements.  The  combined  effects  of  inherent  and  external  R&M 
factors  are  evaluated  at  SDR  to  determine  if  the  contractor's 
proposed  design  is  capable  of  meeting  operational  requirements. 


5.3  PRELIMINARY  DESIGN  REVIEW  (PDR) 

The  PDR  is  an  important  check  on  the  contractor's  progress 
and  consistency  and  the  technical  adequacy  of  the  design  and 
test  approach.  It  is  held  during  FSD  to  evaluate  the 
contractor's  preliminary  design  effort.  The  contractor's 
functional  allocation  of  system  level  requirements  to  individual 
configuration  items  of  hardware  and  software  is  assessed  to 
determine  existing  and  potential  problems  related  to  system 
capability,  equipment  engineering  and  manufacture,  and  logistics 
supportability.  The  R&M  emphasis  at  PDR  is  review  of  the 
contractor's  proposed  R&M  allocations  and  predictions.  Upon 
authentication  of  the  "B"  level  specification  following  PDR, 
these  allocated  R&M  requirements  become  design  requirements 
and  are  included  in  the  system's  allocated  baseline. 
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5.4  CRITICAL  DESIGN  REVIEW  (CDR1 


The  next  major  system  design  milestone  after  the  PDR  is  the 
CDR.  Once  the  contractor  has  successfully  completed  the  PDR,  the 
detailed  design  effort  begins.  Hardware  and  software  are  chosen 
or  designed  to  meet  the  allocated  functions.  At  CDR  the 
contractor's  detailed  design  effort  is  evaluated  against  both 
system  and  functional  specifications.  The  SPO  acts  as  the  single 
interface  between  other  government  offices  and  the  contractor. 

The  CDR  is  held  to  evaluate  the  final  detailed  design  prior 
to  production.  It  will  verify  the  functionality,  producibility, 
and  supportability  of  the  basic  design  and  attempt  to  catch 
oversights  prior  to  the  start  of  production.  Heavy  investments 
are  under  way  at  this  point,  so  the  design  must  be  frozen  and  all 
items  given  to  final  reevaluation  in  order  to  minimize  risk. 

The  R&M  emphasis  at  CDR  is  the  allocation  of  configuration 
item  level  R&M  requirements  down  to  individual  hardware  and 
software  components.  Assessment  of  maintainability  parameters 
against  the  maintenance  concept  and  support  constraints  is  also 
accomplished.  Upon  authentication  of  the  "C"  level 
specifications,  the  system's  product  baseline  is  established. 


6.0  R&M  IN  THE  DEVELOPMENT  PROCESS 


6.1  PROGRAM  MANAGEMENT  REVIEWS  fPMRs^ 

PMRs  are  held  periodically  on  some  programs  to  allow  the 
using,  developing  and  supporting  commands  to  review  the  progress 
of  the  system  acquisition  effort,  define  problems  and  look  for 
solutions.  The  status  of  the  logistics  section  of  the  program 
(including  the  R&M  program)  should  be  reviewed. 


6.2  SECRETARY'S  PROGRAM  REVIEWS  fSPRs^ 

For  designated  programs,  the  SPO  is  regularly  required  to 
report  the  status  of  the  program  to  the  Secretary  of  the  Air 
Force  and  the  Air  Staff.  Included  is  a  section  on  the  status  of 
the  R&M  Program.  Figures  6-1  and  6-2  give  examples  of  the  slide 
format  used  in  the  SPR.  This  material  should  be  reviewed  and 
coordinated  through  HQ  AFSPACECOM/LKYY  before  going  to  the  Air 
Staff. 


6.3  LOGISTICS  SUPPORT  ANALYSIS  (LSA) 

LSA  is  a  subset  of  the  system  engineering  process  in 
which  support  factors  influencing  system  design  and  support 
requirements  are  determined.  R&M  parameters  play  a  major  role  in 
LSA  and  are  tailored  for  each  program.  LSA  tasks  are  contained 
in  MIL-STD-1388-1A;  MIL-STD-1388-2A  describes  a  standard  LSA 
Record  (LSAR)  system.  LSAR  data  sheets  contain  key  R&M 
parameters  at  the  system  as  well  as  the  component  level.  System 
level  R&M  requirements  are  documented  on  the  "A"  sheet  while  the 
B  and  B1  sheets  contain  key  component  level  R&M  data  elements. 

The  LSAR  is  designed  to  serve  as  the  single  point  data  base 
for  logistics  related  design  information.  Reliability 
predictions;  failure  modes,  effects,  and  criticality  analyses; 
and  maintenance  manpower  and  equipment  requirement  determinations 
are  made  from  and  documented  in  the  LSAR.  Using  the  LSAR  as  the 
single  point  data  base  insures  that  analyses  based  on  R&M 
parameters  (e.g..  Life  Cycle  Costing  (LCC) ,  Repair  Level  Analysis 
(RLA) ,  reliability  predictions,  spares  computations,  and  tasks 
analyses)  utilize  the  same  values. 

The  A  sheet  is  completed  during  the  CE/D  phase  as  a  result 
of  LSA  subtask  205.2.3  (specification  requirements)  of 
MIL— STD— 1388— 1A.  It  must  be  available  prior  to,  or  concurrent 
with  initiation  of  LSA  subtask  301  (functional  requirements)  in 
the  CD/V  phase.  The  action  officer  should  obtain  a  copy  of  the  A 
sheet  and  ensure  that  the  logistics  parameters  listed  will 
satisfy  the  SON  R&M  requirements. 
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Since  R&M  and  the  ILS  elements  establish  the  criteria  for 
the  entire  LSA  process,  it  is  important  that  the  A  sheet  gets 
completed  early  on  to  allow  the  up-front  logistics  decisions 
necessary  to  field  a  reliable  and  maintainable  system.  The  SPO 
will  include  information  on  the  application  of  the  LSA  process 
in  the  Integrated  Logistics  Support  Plan  (ILSP) . 


6.4  SYSTEM  PROGRAM  OFFICE  R&M  DOCUMENTATION 

The  SPO  is  responsible  for  writing  two  important  documents 
related  to  R&M  during  system  development:  the  R&M  Management 
Plan  (RMMP)  and  the  ILSP. 


6.4.1.  R&M  Management  Plan  fRMMPl .  The  RMMP  explains  the 
program  management  approach/objectives  and  describes  its  R&M 
program  for  the  acquisition.  The  RMMP  must  relate  to  HQ  USAF  and 
AFSPACECOM  R&M  plans/goals  and  form  the  basis  for  R&M  program 
reviews.  The  action  officer  should: 

(a)  review  the  RMMP  to  verify  that  R&M  has  been 
integrated  into  the  entire  acquisition/ support 
process. 

(b)  follow  up  to  ensure  the  plan  is  being  actively 
executed. 

Additionally,  maintainability  demonstrations  (M  demos)  or 
assessments  are  conducted  to  evaluate  the  testability,  access, 
and  types  of  tools  needed  for  maintenance. 


6.4.2.  Integrated  Logistics  Support  Plan  (ILSP) .  The  ILSP  is 
the  key  document  for  the  overall  logistics  support  of  the  new 
system.  The  ILSP  is  an  expansion  of  section  9  of  the  Program 
Management  Plan  discussed  in  para  3.3.  It's  prepared  IAW  AFR 
800-8,  "Integrated  Logistics  Support  Program."  The  ILSP  covers 
logistics  activities  throughout  the  system's  life  cycle  by 
developing  an  integrated  series  of  individual  plans  covering  the 
different  elements  of  logistics  support.  One  of  these  elements, 
Design  Interface,  includes  the  R&M  program.  The  ILSP  should 
adequately  document  the  status  of  the  R&M  program  and  show  how 
R&M  are  being  used/ integrated  into  the  total  ILS  effort. 


6.5  INTEGRATED  LOGISTICS  SUPPORT  MANAGEMENT  TEAM  MEETINGS 

Integrated  Logistics  Support  Management  Team  (ILSMT) 

Meetings  are  conducted  quarterly.  Normally,  the  ILSMT  Meeting 
is  chaired  by  the  Deputy  Program  Manager  for  Logistics  (DPML)  and 
is  normally  attended  by  representatives  from  the  implementing, 
supporting,  using,  and  test  commands.  AFSPACECOM/ LKY  provides 
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support  to  the  command  managers  at  this  meeting.  R&M  factors 
have  a  significant  impact  on  system  supportability  so  R&M 
progress  should  be  monitored,  problems  discussed  and  cooperative 
action  initiated  at  the  ILSMT  meetings. 


6.6  PREPARATION  FOR  TESTING 


Testing  is  a  critical  function  for  a  system  acquisition. 

In  testing  we  evaluate  a  system  throughout  its  acquisition  cycle 
to  insure  the  final  product  will  meet  system  requirements. 
Preparing  for  this  testing  begins  with  the  establishment  of  a 
Test  and  Evaluation  Master  Plan  (TEMP) .  The  test  methods  used 
to  verify  the  performance  parameters  in  Section  3  of  the  system 
specification  are  documented  in  Section  4  of  the  same 
specification.  The  action  officer  should  ensure  the  test 
methods  and  parameters  for  verifying  R&M  requirements  are  also 
detailed  in  Section  4  of  the  specification  before  it  is 
authenticated.  Establishing  a  Joint  R&M  Evaluation  Team  (JRMET) 
for  each  program  is  also  crucial  to  a  successful  R&M  program. 


6.6.1  Test  and  Evaluation  Master  Plan  (TEMP^ .  The  TEMP  is 
drafted  early  in  the  conceptual  phase  by  the  SPO  and  the 
using/ supporting/testing  commands  to  outline  the  major  elements 
of  the  test  program  (critical  issues,  test  resources,  timing  and 
location) .  The  TEMP  critical  issues  are  usually  separated  into 
two  major  themes:  operational  effectiveness  and  operational 
suitability.  Operational  effectiveness  is  concerned  with  how 
well  the  system  operates  in  its  intended  environment -this  is  the 
operations  part  of  the  test.  Operational  suitability  is 
bpncerned  with  how  well  the  system  can  be  supported  and  whether 
or  not  it  is  ready  to  perform  its  intended  mission-this  is  the 
logistics  support  part  of  the  test.  R&M  are  critical  parts  of 
the  operational  suitability  test  objective  and  should  evaluate  if 
the  system  meets  the  operational  values  contained  in  the  SORD's 
Requirements  Correlation  Matrix  (RCM) .  The  TEMP  should  be 
reviewed/coPjrdinated  by  AFSPACECOM/LKYY  to  insure  it  contains  the 
proper  R&M  parameters . 


6.6.2  Joint  R&M  Evaluation  Team  ( JRMET ) .  The  JRMET  is  held 
regularly  by  the  SPO ' e. Systems  Engineering  Logistics  Branch  to 
review  the  TEMP,  test  plans  and  test  data,  establish  proper  data 
collection  and  data  management  procedures,  correct  deficiencies 
in  data  and,  in  general,  ensure  the  R&M  program  is  successfully 
completed.  The  JRMET  will  draft  a  charter  outlining  its 
functions  and  describing  the  responsibilities  of  each  member. 


AFSPACECOM/LKY  will  normally  attend  these  meetings  as  part  of  the 
logistics  effort.  The  JRMET  should  also  be  attended  by  Command 
engineering  personnel.  The  JRMET  is  a  major  avenue  of  addressing 
and  raising  issues  on  R&M  during  system  acquisition,  since  all 
the  key  players  (SPO,  AFOTEC,  AFLC,  AFSPACECOM  and  the 
contractor)  are  in  attendance. 


7.0  R&M  IN  THE  TESTING  PROCESS 

Section  6.6  discussed  the  importance  of  preparing  for 
testing  and  the  role  of  the  JRMET.  R&M  testing  is  part  of 
the  two  major  test  efforts.  Development  Test  and  Evaluation  and 
Initial  Operational  Test  and  Evaluation  continue  throughout 
FSD, production  deployment,  and  Operational  Support  Phases.  The 
testing  process  reaches  a  high  degree  of  detail  in  system/ 
subsystem  testing,  which  includes  the  support  elements  of  the 
system.  The  test  and  evaluation  tasks  are  the  primary  means  to 
verify  achievement  of  program  objectives  and  justify  the 
continued  or  increased  commitment  of  resources  to  the  program. 

7 . 1  DEVELOPMENT  TEST  &  EVALUATION  (DT&E'l 

DT&E  is  the  responsibility  of  the  SPO.  It  tests  system 
performance  against  system  specifications  to  determine  if  the 
contractor  has  successfully  implemented  the  required  design.  The 
DT&E  test  plan  is  written  by  the  SPO  and  is  supported  by  the 
contractor  prepared/government  approved  DT&E  test  procedures. 
While  the  DT&E  test  environment  is  usually  much  more  restrictive 
than  the  operational  environment,  the  R&M  data  collected  provides 
an  initial  data  base  of  R&M  experience.  This  data  is  passed  to 
the  JRMET  for  review,  classification  and  analysis.  An  initial 
assessment  of  system  R&M  values  is  determined  and  corrective 
action  initiated. 


7.2  OPERATIONAL  TEST  &  EVALUATION  (0T&E> 

OT&E  provides  the  capability  to  track  and  evaluate  the 
system's  initial  operational  R&M  performance;  identify  any 
deficiencies;  and  correct  them  through  changes  in  design, 
technical  data,  software,  support  equipment,  manning,  training, 
or  supply  support.  The  OT&E  process  includes  three  primary 
areas:  test  planning,  test  execution  and  test  reporting. 


7.2.1  Test  Plans.  OT&E  test  plans  are  written  by  AFOTEC  (with 
assistance  from  the  using/developing/supporting  commands)  if 
AFOTEC  is  conducting  the  test.  AFSPACECOM  will  write  the  test 
plan  if  AFOTEC  is  monitoring  the  test.  The  test  plans  should 
show  the  test  objectives,  criteria  and  methodology  that  will  be 
used  to  evaluate  the  R&M  values  outlined  in  the  TEMP.  The  R&M 
values  provide  the  core  of  the  operational  suitability 
evaluation,  so  it  is  important  that  adequate  test  planning  be 
conducted  prior  to  the  test.  Again,  AFS  PACE  COM/ LKY Y  can  provide 
assistance  in  reviewing  the  test  plans  for  consistency  with  the 
test  objectives  and  in  determining  if  the  projected  test  data 
will  provide  adequate  confidence  in  the  test  results. 
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7.2.2  Test  Execution.  Test  execution  is  the  actual  test  period 
during  which  the  data  is  collected  to  make  the  final  test 
report.  The  test  is  normally  conducted  using  the  contractors' 
test  descriptions.  Two  of  the  tests  are  the  reliability  and 
maintainability  demonstrations.  The  reliability  demonstration  is 
conducted  to  test  how  well  the  system  performs  without  failure  in 
the  operational  environment.  The  maintainability  demonstration 
is  conducted  to  evaluate  the  relative  ease  with  which  the  design 
can  be  supported,  verify  maintainability  requirements,  and 
evaluate  the  effectiveness  of  fault  detection/ fault  isolation. 
When  the  FSD  objectives  have  been  attained,  the  program  reaches 
the  production/acceptance  decision. 


7.2.3  Test  Reporting.  The  production/acceptance  decision  is 
supported  by  the  OT&E  test  report.  As  in  7.2.1  above,  whoever 
wrote  the  test  plan  (AFOTEC  or  AFSPACECOM)  will  write  the  test 
report.  The  test  report  details  achieved  R&M,  explains  any 
limitations  that  caused  the  test  data  to  be  less  than  what  was 
desired,  and  identifies  deficiencies  in  the  achieved  R&M  versus 
the  R&M  standards  described  in  the  test  plans. 


8.0  R&M  IN  THE  OPERATIONAL  ERA 

R&M  tracking  continues  into  the  operational  era  with 
Follow-on  OT&E  (FOT&E)  and  maintenance  data  collection  (MDC) . 


8.1  FOLLOW-ON  OPERATIONAL  TEST  &  EVALUATION  ( FOT&E) 

FOT&E  is  conducted  to  test  objectives  not  fully  completed  in 
OT&E,  validate  OT&E  results  and  test  corrections  to  deficiencies 
uncovered  during  OT&E.  The  production  contract  includes  the 
requirement  to  prevent  previously  demonstrated  levels  of  R&M 
performance  from  being  degraded  and  ensure  that  R&M  is  verified 
periodically  during  the  production  run.  Successful  R&M 
demonstration  is  a  condition  of  acceptance  of  production 
articles.  Some  programs  may  contain  Reliability  Improvement 
Warranties  (RIWs)  to  incentivize  production  contractors  to 
improve  system  reliability.  By  correcting  design  problems  or 
defects  that  reduce  system  reliability,  the  contractor  can  reduce 
costs  incurred  by  contract  warranty  clauses. 


8.2  MAINTENANCE  DATA  COLLECTION  CMPCM 

MDC  is  the  system  whereby  the  maintenance  downtime  is 
tracked  and  analyzed  so  that  actual  R&M  performance  can  be 
determined.  AFS PACECOM/ LKM  is  the  OPR  for  MDC.  MDC  should  be 
performed  even  though  the  system  is  contractor  maintained.  This 
allows  AFS  PACECOM  to  evaluate  contractor  as  well  as  system 
performance.  This  data  is  used  to  determine  trends  in 
maintenance  and  project  when  the  system  needs  to  be  replaced  or 
modified.  It's  important  during  this  phase  to  ensure  that 
standardized  (both  in  form  and  method  of  delivery)  MDC 
requirements  are  placed  on  all  contracts  for  maintenance  and 
logistics  support  in  the  operational  era.  The  Air  Force  Standard 
MDC  system  is  the  Core  Automated  Maintenance  System  (CAMS) . 


9 . 0  SUMMARY 


Establishing  realistic  and  consistent  R&M  parameters  in  SONs 
and  SORDs  is  critical  to  fielding  a  system  that  will  meet  our 
operational  requirements  and  be  logistically  supportable.  Your 
responsibility  for  R&M  does  not  stop  with  these  requirements 
documents.  Translation  of  operational  and  logistics  support 
requirements  into  contractual  clauses  is  necessary.  This 
translation  is  a  primary  responsibility  of  the  System  Program 
Office;  the  using  command  is  responsible  for  assisting  in  this 
effort. 

The  R&M  requirements  and  deliverables  specified  in  the 
system  specifications,  statement  of  work,  and  contract  data 
requirements  list  must  be  closely  examined  by  the  using  command. 
Tracking  R&M  allocation  through  specification  and  design  reviews 
is  especially  important. 

Testing  R&M  parameters  is  an  integral  part  of  DT&E  and  IOT&E 
efforts.  DT&E  basically  tests  contract  R&M  parameters;  IOT&E 
tests  operational  R&M  parameters. 

Continued  emphasis  on  and  attention  to  R&M  are  required 
throughout  the  life  cycle  of  the  system.  Once  the  system  is 
fielded,  R&M  data  must  be  collected  through  a  standard 
maintenance  data  collection  system  and  subsequently  analyzed  to 
identify  any  R&M  problems.  Corrective  action  up  to  and  including 
system  replacement  will  be  evaluated.  This  analysis  may  even  be 
used  to  justify  the  preparation  of  a  new  SON/SORD. 
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APPENDIX  A 
R&M  PRIMER 


RELIABILITY  AND  MAINTAINABILITY 
PRIMER 


1.0  INTRODUCTION 

The  purpose  of  this  primer  is  to  provide  the  action  officer 
an  overview  of  the  concepts,  terminology  and  relationships  used 
in  Reliability  and  Maintainability  (R&M) .  It  is  not  designed  to 
make  the  action  officer  an  R&M  engineer.  The  terms  and 
definitions  used  are  taken  from  MIL-STD-721C,  AFSPACECOM/LKY 
Technical  Report  88-1,  and  various  R&M  texts. 

This  primer  starts  with  an  explanation  of  basic  R&M  concepts. 
Next  these  basic  concepts  are  expanded  and  refined  to  include 
mission  and  logistics  R&M  parameters. 


2.0  BASIC  CONCEPTS 

This  section  defines  the  basic  concepts  of  reliability  and 
maintainability,  introduces  item  level  R&M  terms,  and  shows  how 
to  do  rudimentary  R&M  calculations.  These  terms  are  not  to  be 
used  to  state  system  level  operational  R&M  requirements. 

2.1  Reliability  Reliability  is  the  duration  or  probability  of 
failure  free  performance  under  stated  conditions.  It  can 

also  be  defined  as  the  probability  an  item  can  perform  its 
intended  functions  for  a  specified  interval  under  stated 
conditions. 

2.1.1  Mean  Time  Between  Failures  (MTBF)  A  basic  quantitative 
measure  of  reliability  is  Mean  Time  Between  Failures  (MTBF) . 

This  is  the  expected  length  of  time  a  system  will  be  operational 
between  failures.  It  is  normally  calculated  by  taking  the  number 
of  operational  hours  in  a  stated  period  and  dividing  it  by  the 
number  of  failures.  MTBF  could  be  expressed  in  other  life  units 
such  as  number  of  cycles,  number  of  orbits,  or  number  of 
landings. 

2.1.2  Failure  Rate  Failure  rate  is  defined  as  the  number  of 
failures  of  an  item  per  measure  of  life  units  (e.g.,  failures  per 
million  hours,  failures  per  1000  cycles).  The  failure  rate  is 
simply  the  reciprocal  of  the  MTBF.  If  the  average  time  between 
failures  is  1,000,000  hours  (i.e.,  MTBF  =  1,000,000),  then  the 
failure  rate  is  1  failure  per  1,000,000  hours  or  0.000001 
failures  per  hour.  In  some  computations  failure  rates  are 
simpler  to  use  than  the  associated  mean  (average)  value. 

2.1.3  Mission  Reliability  (MR!  The  second  definition  of 
reliability  stated  in  paragraph  2.1  includes  the  added  factor  of 
operating  "for  a  specified  interval".  Instead  of  determining  the 
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average  time  between  failures  or  the  number  of  failures  per  hour, 
Mission  Reliability  (MR)  deals  with  the  probability  of  the  item 
working  continuously  without  a  failure  for  a  specified  period  of 
time.  The  term  most  commonly  used  for  the  specified  time  period 
is  Mean  Mission  Duration.  The  second  factor  affecting  MR  is 
MTBF.  The  mathematical  equation  for  MR  for  most  electronic  items 
is: 

-MMD/MTBF 

*  MR  ■  e 

Where:  e  =  base  of  natural  logarithms  =  2.71828 

MTBF  =  Mission  Time  Between  Failures 
MMD  =  Mean  Mission  Duration 

This  is  the  most  difficult  mathematical  equation  we  will  use  in 
this  primer;  most  hand  held  scientific  or  business  calculators 
will  easily  handle  this  equation.  An  interesting  note  about  MR: 
if  you  design  a  system  with  a  MTBF  equal  to  the  length  of  the 
average  mission  (i.e.,  MTBF  =  MMD),  the  probability  of  surviving 
that  mission  is  only  36.8%.  Mission  Reliability  will  be  further 
\  discussed  in  Section  3.6. 

\  2.2  Maintainability  is  defined  as  a  characteristic  of  a  design 

\that  determines  the  type  and  amount  of  maintenance  required  to 
retain  that  design  in,  or  restore  it  to,  a  specified  condition. 
Maintenance  required  to  retain  an  item  in  its  designed  condition 
is  normally  called  Preventive  Maintenance  (PM)  since  it  prevents 
a  failure  from  occurring.  Maintenance  required  to  restore  an 
item  isxnormally  called  Corrective  Maintenance  (CM)  since  it 
corrects ^some  fault  in  the  system. 

2.2.1  Mean  Time  To  Repair  (MTTR)  The  basic  measure  of  item 
maintainability  is  Mean  Time  To  Repair  (MTTR) .  MTTR  is 
calculated  by  dividing  the  number  of  times  an  item  required 
jrepair  into  the  total  length  of  time  required  to  make  those 
repairs.  MTTR  includes  active  maintenance  time  only.  It  does 
not  include  any  delay  time  (e.g. ,  time  spent  waiting  for  parts) . 

2.2.2  Administrative  and  Logistics  Delay  Time  (ALDT)  There  are 
factors  external  to  the  item  design  that  affect  the  actual  time 
taken  to  perform  maintenance.  Some  of  these  factors  are  the  time 
required  for  the  technician  to  arrive  at  the  failed  item,  time 
spent  waiting  for  tools  or  equipment  to  test  the  item,  and  time 
spent  waiting  for  parts.  The  delays  caused  by  these  external 
factors  are  generically  called  Administrative  and  Logistics  Delay 
Time  (ALDT) . 

2.2.3  Mean  Down  Time  (MPT)  It  is  often  important  to  know  the 
time  required  to  restore  a  failed  item  including  expected  delays. 
Mean  Down  Time  (MDT)  is  the  term  used  for  this.  MDT  is  the  sum 
of  the  active  maintenance  time  plus  administrative  and  logistics 
delays  (i.e.,  MDT  =  MTTR  +  ALDT). 


A 


2 


2.3  Availability  The  probability  an  item  is  available  for  use 
is  a  function  of  how  often  it  breaks  and  how  long  it  takes  to 
repair.  The  formal  definition  of  availability  is  "a  measure  of 
the  degree  to  which  an  item  is  in  an  operable  and  committable 
state  at  the  start  of  a  mission  when  the  mission  is  called  for  at 
any  random  time.  Availability  is  sometimes  referred  to  as  the 
up  time  ratio.  Simplistically  it  is  Uptime/Total  Time  or 
Uptime/ (Uptime  +  Downtime).  Uptime  is  a  measure  of  the  item's 
reliability  and  downtime  a  measure  of  its  maintainability. 

2.3.1  R&M  Trade  Offs  There  are  many  combinations  of  R&M 
values  which  could  yield  the  same  Availability  value.  For 
example  an  item  that  is  up  (operational)  100  hours  and  down 
(failed)  1  hour  has  the  same  availability  as  an  item  up  400  hours 
and  down  4  hours  [100/(100+1)  =  400/(400+4)].  It  may  be 
unacceptable  for  the  item  to  fail  more  frequently  than  every  200 
hours  or  to  take  more  than  four  hours  to  repair.  For  this  reason 
it's  necessary  to  specify  more  than  just  the  item's  availability. 
Figure  2-1  shows  graphically  the  interrelationship  among 
reliability,  maintainability  and  availability. 


2.3.2  Inherent  Availability  (Ai)  Availability  can  refer  to  the 
inherent  capability  of  the  item,  independent  of  the  support 
infrastructure.  This  type  of  availability  is  referred  to  as 
inherent  availability  (Ai) .  The  measure  used  for  Uptime  is 
normally  MTBF.  The  measure  used  for  Downtime  is  MTTR.  The 
resultant  formula  is  Ai  =  MTBF/ (MTBF  +  MTTR) . 
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2.3.3  Operational  Availability  (Ao)  To  determine  the 
availability  expected  in  the  normal  operating  environment,  it 
becomes  necessary  to  account  for  the  impact  of  support  delays. 
This  type  of  availability  is  called  operational  availability 
(Ao) .  The  factor  used  for  Uptime  if  we  assume  there  is  no 
interfering  preventive  maintenance,  is  the  same  one  used  in  Ai 
(i.e.,  MTBF) .  The  Downtime  factor  used  is  Mean  Down  Time  (MDT) 
which  equals  MTTR  +  ALDT.  The  formula  is  Ao  =  MTBF/ (MTBF  +  MDT). 

2.4  Series/Parallel  Models  The  previous  sections  introduced 
the  basic  R&M  concepts  as  applied  to  individual  items.  Items  are 
normally  combined  to  perform  more  complex  functions.  The  way 
they  are  combined  affects  the  reliability  of  that  combination. 
Items  can  be  combined  in  series  or  in  parallel. 

2.4.1  Series  Reliability  When  units  are  combined  in  such  a  way 
that  a  function  must  be  successfully  performed  by  the  first 
unit  before  the  second  unit  can  perform  its  function  and  so  on, 
the  units  are  said  to  be  combined  in  a  series  configuration. 
Figure  2-2  is  a  typical  series  reliability  block  diagram. 


SERIES  RELIABILITY 


R  =  0.90  R  =  0.90  R  =  0.90 


Figure  2-2 


In  figure  2-2,  a  function  requires  items  A,  B,  and  C  to  operate. 
Assume  each  block  has  a  probability  of  operating  of  0.90.  To 
find  the  probability  of  completing  that  function,  we  must 
consider  the  probability  of  A,  B,  and  C  each  working. 
Mathematically  we  do  this  by  multiplying  the  reliability  of  each 
item  (i.e.,  Ra,  Rb,  &  Rc) .  The  formula  for  this  set  of  three 
components  is  "Rs  =  Ra  x  Rb  x  Rc.  In  this  case  the  resultant 
reliability  is  only  72. 9%  (i.e., 0. 9x0. 9x0. 9). 

As  this  example  indicates,  even  though  individual  items  may  have 
good  reliability,  their  serial  combination  results  in  a 
significantly  lowered  reliability. 

2.4.2  Parallel  Reliability  There  is  a  way  to  combine  items  so 
that  the  combined  reliability  is  better  than  the  individual 
reliabilities.  When  items  are  tied  together  so  their  function 
can  be  performed  by  any  one  (or  more)  of  them,  they  are  said  to 


4 


A 


In  Figure  2-3,  as  indicated  by  the  1/3,  only  one  of  the  three 
items  must  be  working  for  the  function  to  be  performed.  In  other 
words  the  function  fails  only  when  all  three  items  fail 
simultaneously.  We  can  find  the  probability  of  this  happening  by 
multiplying  the  individual  probabilities  of  failing. 

Let's  assume  items  D,E,  and  F  each  have  a  probability  of  not 
failing  (Reliability)  of  0.90.  Since  the  probability  of  failing 
added  to  the  probability  of  not  failing  must  equal  one,  the 
probability  of  failing  equals  0.10,  (i.e.,  1  -  0.9).  The  chance 

of  all  three  items  failing  is  0.1  x  0.1  x  0.1  =  .001.  Since  we 
now  know  the  probability  of  all  three  failing,  we  subtract  it 
from  one  to  get  the  probability  of  one  or  more  not  failing  (1.0  - 

0.001  =  0.999).  So  in  this  configuration,  the  probability  of  the 

function  being  performed  is  0.999. 

Putting  this  logic  into  a  formula  we  get: 

If  reliability  of  D  is  Rd  then  its  unreliability  is  1  -  Rd 

If  reliability  of  E  is  Re  then  its  unreliability  is  1  -  Re. 

If  reliability  of  F  is  Rf  then  its  unreliability  is  1  -  Rf . 

The  probability  of  D,  E,  &  F  failing  is  (1-Rd) (1-Re) (1-Rf) . 
The  probability  of  D,  E,  &  F  not  failing  (Rs)  is 

Rs  -  1  -[ (1-Rd) (1-Re) (1-Rf) ] 

It's  also  possible  to  put  items  in  parallel  and  require  more  than 
one  to  remain  operational.  Computing  the  probability  of  the 
function  being  performed  requires  the  use  of  complex  mathematical 
computations  outside  the  scope  of  this  primer. 

2.4.3  Series/Parallel  Combinations  In  more  complex 
configurations  there  are  often  combinations  of  series  and 
parallel  items.  In  these  cases  it  is  relatively  simple  to  reduce 
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3.0  MISSION  R&M 

The  concepts  introduced  in  section  two  are  basic, 
component  level  concepts.  These  concepts  can  be  applied  to 
system  level  requirements  with  some  minor  modifications  and  _ 
expansions.  This  section  will  define  the  R&M  terms  for  use  in 
defining  operational  requirements  in  Statements  of  Operational 

Need  (SONS)  and  System  Operational  Requirements  Documents 

(SORDs) .  These  terms  are  especially  adaptable  for  use  with 
space,  space  surveillance  and  missile  warning  systems. 

3.1  Mission  Effectiveness  (ME)_  System  capability  has  two 
factors  associated  with  it.  The  first  factor  is  the  probability 
of  a  system  being  operable  and  capabie  of  initiating  a  missio: n  _ at 
any  (random)  time.  The  second  factor  is  the  probability  that 
system  is  operable  and  capable  at  any  (random)  timeduring  a  _ 
mission.  If  these  two  probabilities  are  expressed  in  consisten 
terms,  then  the  probability  of  effectively  completing  the  mission 
is  the  product  of  the  two  factors. 

The  first  factor  is  the  traditional  definition  of  system  _ 
availability.  The  second  factor  is  a  measure  of  how  dependable  a 
system  is  once  a  mission  has  begun.  In  this  context, 
availability  is  associated  with  non-mission  time  and 
dependability  with  mission  time.  Since  the  system  ^frequently 
inactive  or  exercised  differently  during  nonTmission  time,  there 
are  different  non-mission  and  mission  reliability  and  . 

maintainability  factors  associated  with  it.  Figure  3-1  shows  the 
relationships  among  these  factors. 

SYSTEM  TIME  VERSUS  R&M  PARAMETERS 


MISSION  EFFECTIVENESS 
(TOTAL  TIME) 


AVAILABILITY 
(NON-MISSION  TIME) 


INACTIVE  TIME  ACTIVE  TIME 


DEPENDABILITY 
(MISSION  TIME) 

ACTIVE  TIME 


Figure  3-1 


3  2  Availability  &  Dependability  The  terms  availability  and 
dependability  are  not  new.  These  terms  are  listed  in 
MiL-STD-7 2 1C ,  Definition  of  Terms  for  Reliability  and 
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Maintainability.  Although  the  term  availability  has  always  been 
defined  as  the  probability  of  being  operable  and  capable  at  any 
(random)  time  of  initiating  a  mission,  it  has  often  been  used  to 
calculate  the  probability  of  the  system  working  at  any  (random) 
time  during  a  mission. 

The  terms  and  concepts  of  availability  and  dependability  are 
also  well  documented  in  standard  logistics  engineering  and 
reliability  texts.  Examples  are  Benjamin  S.  Blanchard's  2nd 
edition  of  Logistics  Engineering  and  Management.  Page  20,  and 
Igor  Bazovsky's  Reliability  Theory  and  Practice.  Chapter  17. 

Misuse  of  the  term  availability  hasn't  created  many  problems  in 
the  aircraft  world  because  frequently  only  mission  time  is 
involved.  However,  if  we  are  to  effectively  account  for  the 
unique  R&M  parameters  associated  with  spacecraft  launch,  on  orfoi 
mission,  and  space  vehicle  turn  around,  a  return  to  the  more 
accurate  definitions  is  required. 

3*3  Mission  Time  R&M  Parameters  The  R&M  parameters  associated 
with  non-mission  time  can  be  inherently  different  from  those  with 
mission  time  (e.g.,  Age  deterioration  of  seals  versus  wearout 
failures) .  MIL-STD-721C  defines  the  reliability  and 
maintainability  parameters  associated  with  dependability  (mission 
time)  as  Mission  Time  Between  Critical  Failures  (MTBCF)  and 
Mission  Time  To  Restore  Functions  (MTTRF) .  As  their  titles 
indicate,  they  are  only  concerned  with  the  impact  on  mission. 

This  makes  them  ideal  for  stating  operational  requirements  and 
for  documenting  results  of  operational  testing. 

3.3.1  Mission  Time  Between  Critical  Failures  (MTBCF)  The 
definition  of  Mission  Time  Between  Critical  Failures  from 
MIL-STD-721C  is:  A  measure  of  mission  reliability.  The  total 
amount  of  mission  time,  divided  by  the  total  number  of  critical 
failures  during  a  stated  series  of  missions.  Its  formula  is: 

TOTAL  MISSION  TIME 

MTBCF  - - - - 

#  OF  CRITICAL  FAILURES 

3.3.2  Mission  Time  To  Restore  Functions  f MTTRF ^  The  definition 
of  Mission  Time  To  Restore  Functions  is:  A  measure  of  mission 
maintainability.  The  total  maintenance  time  required  to  restore 
mission  functions  interrupted  by  critical  failures  divided  by  the 
number  of  critical  failures  during  the  course  of  a  specified 
mission  profile.  MTTRF  includes  both  scheduled  and  unscheduled 
maintenance.  Its  formula  is: 

TOTAL  RESTORAL  TIME 

MTTRF  = - - - - — - 

#  OF  CRITICAL  FAILURES 

When  MTTRF  is  used  to  calculate  Operational  Dependability  (Do) 
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versus  Inherent  Dependability  (Di) ,  Administrative  and  Logistics 
Delay  Time  (ALDT)  is  included  as  part  of  total  restoral  time. 

3.3.3  Maximum  Mission  Time  To  Restore  Functions _ (MaxTTRF)  If 

only  the  mean  MTTRF  value  is  specified,  there  may  be  a  wide  range 
of  individual  repair  time  values.  Figure  3-2  shows  two  repair 
time  distributions  with  the  same  mean  value.  The  lower  curve  has 
a  greater  variation  around  the  mean  value.  Two  systems  could  be 
designed  to  meet  the  same  mean  MTTRF  specification  and  not 
necessarily  meet  operational  requirements.  A  system  design  which 
follows  the  lower  distribution  will  have  more  failures  that  take 
longer  to  repair  than  one  whose  design  fits  the  upper  curve. 

This  could  result  in  repairs  taking  longer  than  is  operationally 
acceptable,  on  too  frequent  a  basis. 


REPAIR  TIME  DISTRIBUTIONS  WITH  THE  SAME  MEAN 
BUT  WITH  DIFFERENT  VARIANCE 

P(t) 


Because  of  this,  it  may  become  necessary  to  place  restrictions  on 
the  maximum  amount  of  time  the  system  must  be  restored  within. 
This  can  be  done  by  specifying  a  Maximum  Time  To  Restore 
Functions  (MaxTTRF)  at  a  stated  percentile.  For  example,  you 
could  specify  that  90%  of  all  repairs  be  accomplished  in  2  hours 
or  less.  This  parameter  is  sometimes  referred  to  as  Maximum 
Continuous  Downtime. 

3.3.4  MaxTTRF  and  Availabilitv/Dependability  As  seen  earlier  in 
this  primer,  availability  and  dependability  are  normally  based  on 
mean  values.  If  we  know  the  maximum  time  we  want  the  system 
repaired  in,  we  need  a  way  to  find  the  mean  value  for  use  in 
these  calculations.  There  is  a  relationship  between  the  mean  and 
maximum  values  for  a  given  distribution.  By  knowing  what  the 
underlying  distribution  is,  we  can  calculate  the  mean  value.  For 
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more  information  on  how  to  convert  mean  and  max  values,  see 
Section  3.5  on  distributions. 

3.4  Non-mission  Time  R&M  Parameters  During  non-mission  time  it 
is  possible  to  have  active  and  inactive  time.  There  can  be 
inherent  characteristics  of  the  system  that  affect  R&M  parameters 
during  these  times.  Under  certain  circumstances,  it  may  be 
logical  to  separate  the  R&M  factors  associated  with  inactive 
non-mission  and  active  non-mission  time.  However,  under  normal 
circumstances  it  usually  is  adequate  to  address  non-mission  time 
R&M  factors  as  a  single  entity. 

3.4.1  Mean  Time  Between  Downing  Events  (MTBDE)  A  downing  event 
is  any  event  during  non-mission  time  that  affects  the  system's 
ability  to  initiate  a  mission.  Scheduled  interfering  preventive 
maintenance  or  servicing  actions  required  to  maintain  the  system 
in  a  state  capable  of  initiating  a  mission  are  examples  of 
downing  events.  MIL-STD-721C  calls  MTBDE  the  reliability 
parameter  associated  with  readiness. 

The  definition  of  MTBDE  is:  A  measure  of  system  reliability 
associated  with  availability.  The  total  non-mission  time  divided 
by  the  #  of  downing  events.  It  formula  is: 

TOTAL  NON-MISSION  TIME 

MTBDE  =  — - 

#  OF  DOWNING  EVENTS 

3.4.2  Mean  Time  To  Restore  System  (mttrs)  The  collateral 
maintainability  criteria  associated  with  readiness  is  Mean  Time 
To  Restore  System  (MTTRS) .  MTTRS  applies  only  to  maintenance 
actions  that  occur  during  non-mission  time.  MTTRS  includes  both 
scheduled  and  unscheduled  maintenance  actions. 

The  definition  of  MTTRS  is:  A  measure  of  system  maintainability 
associated  with  availability.  The  total  maintenance  time 
associated  with  downing  events  divided  by  the  number  of  downing 
events.  Its  formula  is: 

Total  Restoral  Time 

MTTRS  =  - 

#  of  Downing  Events 

When  MTTRS  is  used  to  calculate  Operational  Readiness  (Ro)  versus 
Inherent  Readiness  (Ri) ,  total  restoral  time  includes 
Administrative  and  Logistics  Delay  Time  (ALDT) . 

3.5  Distributions  Failure  rates  and  repair  times  tend  to  follow 
a  general  pattern  for  specific  types  of  systems.  The  general 
pattern  followed  fits  a  specific  statistical  distribution. 

3.5.1  Failure  Rate  Distributions  There  are  three  basic  types  of 
failures  for  communication-electronic  systems.  These  are  burn-in 
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failures,  random  failures  and  wear  out  failures.  Burn-in 
failures  are  normally  screened  out  by  testing  or  environmental 
stress  screening  prior  to  placing  the  system  in  field  operation. 
During  the  burn-in  period,  the  failure  rate  normally  decreases; 
random  failures  occur  during  the  useful  life  period.  The  failure 
rate  during  the  useful  life  period  remains  relatively  constant. 
During  the  wearout  period  the  failure  rate  increases.  These 
three  types  of  failure  rates  are  shown  graphically  in  Figure  3-3. 
This  particular  curve  is  commonly  known  as  the  bathtub  curve. 


COMPONENT  FAILURE  RATES  AS  A  FUNCTION  OF  TIME 


Figure  3-3 


Since  burn-in  failures  normally  occur  during  the  development/ 
production  phase,  we  are  only  concerned  about  random  and  wearout 
failures  in  this  primer. 

3. 5. 1.1  Exponential  Failure  Distribution  Failures  during  the 
useful  life  period  for  communication-electronic  components  follow 
an  exponential  distribution.  With  this  type  of  distribution, 

63.2  percent  of  the  failures  will  occur  before  the  mean  value. 
From  another  perspective  the  mean  value  will  be  met  or  exceeded 
only  36.8  percent  of  the  time.  Figure  3-4  is  a  graph  of  the 
exponential  probability  density  function. 

3.5. 1.2  Normal  Failure  Distribution  Component  failures  caused 
by  wearout  follow  a  normal  distribution.  Figure  3—5,  shows  the 
normal  wearout  of  a  lamp.  The  average  life  expectancy  for  this 
particular  lamp  is  7200  hours.  A  constant  level  of  random 
failures  would  be  expected  during  the  useful  life  period.  Once 
the  useful  life  period  is  over  (around  5000  hours) ,  the  lamps 
will  start  experiencing  wearout  failures.  By  7200  hours  we  would 
expect  half  the  lamps  to  have  failed.  The  other  half  would  fail 
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after  7200  hours.  The  rate  of  failure  would  be  symmetrical 
around  the  mean  value. 


3.5.2  Restoral  Time  Distributions  Like  failure  rates,  restoral 
times  also  tend  to  follow  specific  patterns  or  distributions.  For 
communication-electronic  systems  three  types  of  distributions  are 
commonly  found.  The  type  of  maintenance  actions  required  to 
restore  the  system  affects  which  of  the  three  is  appropriate. 


3. 5. 2.1  Exponential  Repair  Distributions  When  system  restoral  is 
primarily  accomplished  by  a  combination  of  manual  and  automatic 
switchover  to  a  redundant  unit,  an  exponential  distribution  is 


normally  followed.  This  curve  is  shown  in  Figure  3-6.  As  seen 
from  this  graph,  the  majority  of  items  can  be  repaired  within  a 
short  restoral  time  with  fewer  restoral  actions  occurring  as 


repair  time  increases.  As  with  the  exponential  failure  rate 
curve,  62.8  percent  of  the  repair  actions  will  be  accomplished 
before  the  mean  repair  time  is  reached. 


Mean  (MDT)  and  maximum  (Mmax)  allowable  downtimes  are  shown  in 
Figure  3-6.  The  0.69  sigma  indicates  the  number  of  standard 
deviations  the  mean  is  away  from  the  origin  (time  zero).  The  2.30 
sigma  is  the  number  of  standard  deviations  the  90th  percentile  is 
away  from  the  origin.  The  hatched  area  indicates  repair  times 
that  are  equal  to  or  less  than  the  maximum  allowable  down  time. 

From  this  information  we  can  establish  a  ratio  between  the  mean 
and  maximum  values.  For  instance,  if  we  want  90  out  of  100 
repair  actions  to  be  accomplished  in  1  hour  or  less,  a  mean  value 
of  0.30  hours  [(0.69/2.30)  (1  hour)]  is  expected.  Similarly,  if 


A 


13 


we  specify  a  mean  value  of  1  hour,  90  out  of  100  repair  actions 
will  be  accomplished  within  3.33  hours  [(2.30/0.69)  (1  hour)]. 

3 . 5 . 2 . 2  Log-Normal  Repair  Distributions  When  system  restoral  is 
accomplished  primarily  by  a  combination  of  switchover  and 
on-equipment  repair,  a  Log-Normal  distribution  is  often  seen. 
Figure  3-7  depicts  a  typical  Log-Normal  probability  density 
function  with  mean  (MDT)  and  maximum  (Mmax)  allowable  downtime 
shown.  The  0.8  sigma  indicates  the  number  of  standard  deviations 
the  mean  is  away  from  the  origin  (time  zero) .  The  1.86  sigma  is 
the  number  of  standard  deviations  the  90th  percentile  is  away 
from  the  origin.  The  hatched  area  indicates  repair  times  that 
are  equal  to  or  less  than  the  maximum  allowable  down  time. 


From  this  information  we  can  establish  a  ratio  between  the  mean 
and  maximum  values.  For  instance,  if  we  want  90  out  of  100 
repair  actions  to  be  accomplished  in  1  hour  or  less,  a  mean  value 
of  0.43  hours  [(0.80/1.86)  (1  hour) ]  is  expected.  Similarly,  if 

we  specify  a  mean  value  of  1  hour,  90  out  of  100  repair  actions 
will  be  accomplished  within  2.325  hours  [(1.86/0.80)  (1  hour)]. 

3. 5. 2. 3  Normal  Repair  Time  Distribution  When  system  restoral  is 
accomplished  primarily  by  on-equipment  repair  actions  with 
associated  administrative  and  logistics  delays,  a  normal 
distribution  is  usually  found.  As  with  the  normal  failure  rate 
distribution,  50  percent  of  the  repair  actions  will  be 
accomplished  before  the  mean  value  and  50  percent  after.  The 
normal  probability  density  function  including  mean  (MDT)  and 
maximum  (Mmax)  allowable  downtime  for  repair  actions  is  shown  in 
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Figure  3-8.  The  3.0  sigma  indicates  the  number  of  standard 
deviations  the  mean  is  away  from  the  origin  (time  zero).  The 
4.28  sigma  is  the  number  of  standard  deviations  the  90th 
percentile  is  away  from  the  origin.  The  hatched  area  indicates 
repair  times  that  are  egual  to  or  less  than  the  maximum  allowable 
down  time.  As  with  the  exponential  and  log-normal  distributions, 
we  can  establish  a  ratio  between  the  mean  and  maximum  values.  For 
instance,  if  we  want  90  out  of  100  repair  actions  to  be 
accomplished  in  1  hour  or  less,  a  mean  value  of  0.70  hours 
[(3.00/4.28)  (1  hour)]  is  expected.  Similarly,  if  we  specify  a 

mean  value  of  1  hour,  90  out  of  100  repair  actions  will  be 
accomplished  within  1.43  hours  [(4.28/3.00)  (1  hour)]. 


3.5.3  Mean/Maximum  Conversions  If  operational  considerations 
require  a  specified  maximum  allowable  downtime,  the  above 
transformations  become  especially  useful.  It  is  possible  to 
specify  percentile  values  other  than  the  90th  percentile  but 
usually  not  feasible  to  design  a  system  so  that  100%  of  all 
possible  failures  can  be  repaired  in  a  stated  period.  The  90th 
percentile  is  normally  used  because  it  represents  an  efficient 
tradeoff  between  cost  and  repair  time. 

AFOTEC  has  collected  and  analyzed  repair  time  data  from  a 
multitude  of  communication-electronic  systems.  After  applying 
curve  fitting  techniques,  they  have  found  that  the  log-normal 
distribution  typifies  these  systems.  With  this  assumption  we  can 
make  a  reasonable  approximation  of  the  mean  value  associated  with 
specific  maximum  values. 
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.3 . 6  Mission  Reliability  (MR) 

The  concept  of  mission  reliability  that  was  discussed  in  the 
basic  concepts  section  can  be  applied  directly  at  the  system 
level.  Instead  of  MTBF,  the  term  Mission  Time  Between  Critical 
Failures  (MTBCF)  is  used.  Mission  reliability  is  a  measure  of 
system  reliability  during  mission  time.  It  does  not  take  into 
account  system  maintainability  or  availability  factors. 

There  are  many  missions  and  systems  which  do  not  allow  restoral 
of  specific  functions  during  the  mission.  Consider  the  example 
of  a  spacecraft  oxygen  or  propulsion  system.  If  a  critical 
failure  occurs,  it  may  not  be  possible  to  restore  the  system 
prior  to  running  out  of  oxygen  or  in  time  to  achieve  orbit.  It 
then  becomes  an  operational  requirement  to  achieve  a  stated 
reliability  for  a  stated  mission  duration. 

The  definition  of  Mission  Reliability  (MR)  is:  A  measure  of  the 
degree  to  which  a  system  is  operable  and  capable  of  performing 
its  required  functions  at  a  specified  time  into  a  mission  or  for 
a  mission  of  stated  duration.  It  is  based  on  the  effects  of 
mission  reliability  during  mission  time  only. 

The  concept  of  mission  reliability  is  based  on  the  mathematical 
probability  of  survival  function.  The  formula  for  mission 
survivability  will  vary  based  on  the  underlying  distribution  of 
mission  critical  failures.  For  an  exponential  distribution  the 
formula  for  MR  is: 

-MMD/MTBCF 

MR  =  e 

Where:  e  =  base  of  natural  logarithms 

MTBCF  =  Mission  Time  Between  Critical  Failures 
MMD  =  Mean  Mission  Duration 


3.7  Mission  Versus  Logistics  Parameters  The  R&M  parameters 
discussed  above  deal  with  the  effects  on  mission  accomplishment. 
They  are  of  prime  importance  to  the  operational  commands.  The 
operational  commands  are  also  very  concerned  with  a  second  set  of 
R&M  parameters  that  deal  with  logistics  support  related  terms. 
These  parameters  have  significant  impact  on  command  manpower  and 
supply  support.  Section  4  of  this  primer  addresses  these 
factors. 
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4 . 0  LOGISTICS  R&M 

Although  mission  R&M  parameters  are  an  indication  of  the 
capability  of  the  system  to  perform  specified  mission  profiles, 
they  are  not  the  only  ones  operational  commands  are  concerned 
with.  This  section  will  address  the  R&M  parameters  associated 
with  logistics  support  and  their  impact  on  the  operational 
command . 

4 . 1  Maintenance  Concerns  The  operational  command  is  concerned 
with  R&M  factors  that  affect  maintenance.  Of  special  interest 
are  the  areas  of  maintenance  manpower  and  maintenance  cost.  These 
parameters  may  be  used  in  requirements  documents  when  they 
specify  operational  constraints  the  system  must  be  designed 
within.  An  example  is  a  system  that  must  be  designed  to  be 
maintained  with  the  same  or  fewer  number  of  personnel  as  the 
system  it  replaces  due  to  manpower  ceiling  limitations.  Another 
example  is  a  contractor  maintained  system  whose  Operations  and 
Maintenance  (O&M)  budget  must  not  exceed  programmed  contract 
maintenance  funds. 

4.2  Mean  Time  Between  Maintenance  (MTBM)  Regardless  of  the 
impact  on  mission  effectiveness,  every  maintenance  action  can 
have  an  impact  on  maintenance  manning.  The  number  of  maintenance 
actions  and  the  length  of  time  to  perform  the  average  maintenance 
action  combine  to  determine  the  basic  manpower  requirement.  The 
reliability  term  associated  with  maintenance  manning  is  Mean  Time 
Between  Maintenance  (MTBM) . 

Since  the  purpose  of  this  term  is  to  determine  the  impact  on 
maintenance  manpower  and  cost,  all  maintenance  actions  should  be 
considered.  This  includes  mission/non-mission  time  as  well  as 
scheduled/unscheduled  maintenance  actions. 

The  definition  of  MTBM  is:  A  measure  of  system  reliability 
taking  into  account  maintenance  policy.  The  total  number  of 
system  life  units  in  a  stated  period  of  time  divided  by  the 
number  of  maintenance  events  both  scheduled  and  unscheduled.  The 
formula  is: 


Since  the  frequency  of  MTBSM  and  MTBUM  may  vary,  it  is  necessary 
to  convert  then  to  rates,  add  them,  and  take  the  reciprocal. 
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4.2.1  Mean  Time  Between  Scheduled  Maintenance  (MTBSM1  Mean  Time 
Between  Scheduled  Maintenance  is  the  term  used  to  indicate  the 
average  frequency  of  scheduled  maintenance  events.  The  term 
'•preventive  maintenance"  is  frequently  used  to  denote  these  types 
of  maintenance  events.  Preventive  maintenance  has  the 
connotation  of  actions  taken  specifically  to  prevent  a  failure. 

On  certain  space  and  missile  warning  systems,  routine  servicing 
activities  (e.g.,  refueling  of  a  satellite)  may  occur.  In  some 
respects  these  servicing  actions  are  not  preventive  maintenance 
actions.  For  this  reason,  the  use  of  the  more  generic  term 
scheduled  maintenance  is  proposed. 


The  definition  of  Mean  Time  Between  Scheduled  Maintenance  (MTBSM) 
is:  A  measure  of  system  reliability  taking  into  account 
maintenance  policy.  The  total  relevant  system  time  divided  by 
the  number  of  scheduled  maintenance  events.  The  formula  for 
MTBSM  is : 


MTBSM  - 


TOTAL  SYSTEM  TIME 
#  OF  SCHEDULED  MAINTENANCE  EVENTS 


4.2.2  Mean  Time  Between  Unscheduled  Maintenance  (MTBUM1  Mean 
Time  Between  Unscheduled  Maintenance  is  the  term  used  to  denote 
the  average  frequency  of  unscheduled  maintenance  events.  The 
term  unscheduled  maintenance  is  proposed  instead  of  corrective 
maintenance  for  two  reasons.  The  first  reason  is  that  certain 
preventive  maintenance  events  (e.g.,  repainting  for  corrosion 
control)  are  done  on  an  as  required  (i.e.,  unscheduled)  basis. 
The  second  reason  is  to  be  consistent  with  the  term  scheduled 
maintenance. 


The  definition  of  Mean  Time  Between  Unscheduled  Maintenance 
(MTBUM)  is  "  A  measure  of  system  reliability  taking  into  account 
maintenance  policy."  The  total  relevant  system  time  divided  by 
the  number  of  unscheduled  maintenance  events.  The  formula  for 
MTBUM  is: 


MTBUM  = 


TOTAL  SYSTEM  TIME 

#  OF  UNSCHEDULED  MAINTENANCE  EVENTS 


4.3  Mean  Maintenance  Time  (MMT)  While  MTBM,  MTBSM,  and  MTBUM 
are  measures  of  system  reliability,  each  have  a  collateral 
measure  of  system  maintainability.  The  collateral 
maintainability  term  for  MTBM  is  Mean  Maintenance  Time  (MMT) . 

The  definition  of  MMT  is:  A  measure  of  system  maintainability 
considering  maintenance  policy.  The  sum  of  unscheduled  and 
scheduled  maintenance  times  divided  by  the  sum  of  scheduled  and 
unscheduled  maintenance  events  during  a  stated  period  of  time. 
The  formula  is : 
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SCHEDULED  +  UNSCHEDULED  MAINTENANCE  TIME 

MMT  - - * - - - 

#  OF  SCHEDULED  +  UNSCHEDULED  MAINTENANCE  EVENTS 

4.3.1  Mean  Scheduled  Maintenance  Time  (MSMT)  The 
maintainability  parameter  associated  with  Mean  Time  Between 
Scheduled  Maintenance  (MTBSM)  is  Mean  Scheduled  Maintenance  Time 
(MSMT) .  The  definition  of  MSMT  is:  A  measure  of  system 
maintainability  taking  into  account  maintenance  policy.  Total 
scheduled  maintenance  time  divided  by  the  number  of  scheduled 
maintenance  events  during  a  stated  period  of  time.  The  formula 
is: 

TOTAL  SCHEDULED  MAINTENANCE  TIME 

MSMT  =  - - 

#  OF  SCHEDULED  MAINTENANCE  EVENTS 

4.3.2  Mean  Unscheduled  Maintenance  Time  (MUMT)  The 
maintainability  parameter  associated  with  Mean  Time  Between 
Unscheduled  Maintenance  (MTBUM)  is  Mean  Unscheduled  Maintenance 
Time  (MUMT) .  The  definition  of  MUMT  is:  A  measure  of  system 
maintainability  considering  maintenance  policy.  Total 
unscheduled  maintenance  time  divided  by  the  number  of  unscheduled 
maintenance  events.  The  formula  is: 

TOTAL  UNSCHEDULED  MAINTENANCE  TIME 

MUMT  =  - 

#  OF  UNSCHEDULED  MAINTENANCE  EVENTS 

4.4  Supply  Concerns  The  operational  command  is  also  concerned 
with  R&M  parameters  that  affect  supply  support.  The  frequency  of 
demands  on  the  supply  system  and  the  cost  of  that  support  are  two 
areas  of  special  interest.  Although  supply  R&M  parameters  are 
normally  used  to  assess  support  cost  of  a  specified  design,  they 
can  also  be  used  to  state  operational  constraints.  Mission 
deployment  requirements  may  limit  the  demands  that  can  be  placed 
on  supply.  Budget  limitations  may  require  the  supply  support 
costs  to  be  within  a  programmed  amount  (e.g.,  Supply  support 
contract  dollars  are  limited) . 

4.4.1  Mean  Time  Between  Demand  (MTBD)  Regardless  of  the  impact 
.  on  mission  effectiveness,  every  demand  on  supply  affects  supply 
support  cost.  The  cost  of  this  support  must  be  programmed, 
planned  and  budgeted.  The  average  frequency  of  demands  and  the 
average  cost  of  each  demand  are  used  to  assist  in  determining  the 
amount  to  be  funded.  The  measure  of  the  reliability  related  to 
the  frequency  of  demands  placed  on  supply  is  Mean  Time  Between 
Demand  (MTBD) .  The  definition  of  MTBD  is:  A  measure  of  the 
system  reliability  parameter  related  to  supply  support.  The 
total  number  of  system  life  units  divided  by  the  total  number  of 
items  demanded  from  supply  during  a  stated  period  of  time.  The 
formula  for  MTBD  is: 
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MTBD 


TOTAL  LIFE  UNITS 


#  OF  ITEMS  DEMANDED 


4.4.2  Mean  Parts  Cost/Demand  (MPC/D)  Once  the  frequency  of 
demands  and  the  average  cost  of  each  demand  is  known,  supply 
support  costs  can  be  estimated.  The  term  associated  with  the 
cost  of  demands  is  Mean  Parts  Cost/Demand  (MPC/D) .  The 
definition  of  MPC/D  is:  A  measure  of  system  support  costs 
related  to  supply  reliability.  The  total  cost  of  parts  demanded 
from  supply  divided  by  the  number  of  demands  during  a  stated 
period  of  time.  The  formula  is: 

TOTAL  PARTS  COSTS 

MPC/D  - - - - 

#  OF  DEMANDS 


A 


20 


APPENDIX  B 


SYSTEM  OPERATIONAL  REQUIREMENTS  DOCUMENT  CHECKLIST 


SQRD  CHECKLIST 


s 


1.  Does  the  SORD  address,  as  a  minimum,  these  Reliability 

and  Maintainability  factors: 

a.  What  is  the  required  system  R&M  effectiveness? 

b.  Based  on  system's  employment  and  deployment  concepts 
what  is  the  system's  required  availability? 

c.  What  is  the  required  dependability  once  a  mission  is 
initiated? 

d.  What  is  required  mission  reliability  for  a  mission  of 
a  stated  duration? 

e. ;  What  is  the  maximum  acceptable  system  restoral  time? 

f.  What  frequency  of  critical  software  failures  is 
acceptable? 

g.  What  frequency  of  hardware  failures  is  acceptable? 

h.  What  maintenance  and  operations  manpower  constraints 
will  the  system  be  required  to  be  operated  in. 


2 .  Do  the  above  parameters  reflect  improved  R&M  system 

|  performance  parameters  over  the  system (s)  being  replaced? 

'3.  Concerning  maintenance,  does  the  SORD  address 

a.  Levels  of  maintenance? 

b.  Skill  level  of  blue-suit,  robotic  or  contractor 
^personnel  designated  to  maintain  the  system? 

'■  *  J  ,  .  *  t 

4.  Does?  the  SORD  specify ^quantitative  values  for  operational 
and  logistics  support  performance  parameters? 

5.  Do  quantitative  and  qualitative  readiness  requirements 
reflect  the  command  R&M  goals? 

•  .i- 

6.  Are  the  R&M  terms  integrated  such  that: 

a.  R&M  parameters  are  explained  in  terms  of  how  they 
affect  operational  capability? 

b.  R&M  terms  used  in  the  SORD  are  defined  sufficiently 

for  conversion  into  contractual  terms  in  a  future 
System  Specification?  V 


7. 


c.  R&M  goals  and  requirements  are  reasonable  and 
compatible? 

Are  values  for  built-in  test  equipment  (BITE) 
effectiveness  defined? 

8.  Is  the  level  of  diagnostics  defined? 

Has  software  R&M  been  addressed? 


9. 


APPENDIX  C 


R&M  DATA  ITEM  DESCRIPTION  LIST 


RELIABILITY  DATA  ITEM  DESCRIPTIONS  (DID) 

The  following  is  a  list  of  data  item  descriptions  associated  with 
the  reliability  tasks  specified  in  MIL-STD-785B. 


TASK 

APPLICABLE  DID 

DATA  REQUIREMENT 

101 

DI-R-7079 

Reliability  Program  Plan 

103 

DI-R-7  080 

Reliability  Status  Report 

104 

DI-RELI-80255 

Report,  Failure  Summary  and  Analysis 

201 

DI-R-7081 

Reliability  Mathematical  Models 

202 

DI-R-2114 

Report,  Reliability  Allocation 

203 

DI-R-7082 

Reliability  Predictions  Report 

204 

DI-R-7  085A 

Report,  Failure  Modes,  Effects  and 
Criticality  Analysis  Report  (FMECA) 

DI-R-7  086 

FMECA  Plan 

205 

DI-R-7083 

Sneak  Circuit  Analysis  Report 

206 

DI-R-7  084 

Electronic  Parts/Circuits  Tolerance 
Analysis  Report 

208 

DI-R-30511 

Plan,  Critical  Item  Control 

The  following  tasks  have  DIDs  associated  with  them  related  to 
imposition  of  MIL-STD-781C: 

301 

DI-R-7 040 

Report,  Burn-in  Test 

302, 

303, 
304 

DI-RELI-80250 

Plan,  Reliability  Test 

303, 

304 

DI-RELI-80251 

Procedures,  Reliability  Test  and 
Demonstration 

C-l 


303, 

304  DI-RELI-80252  Reports,  Reliability  Test  and 

Demonstration  (Final  report) 


Notes:  (1)  Only  data  items  specified  in  the  CDRL  are 

deliverable.  Therefore,  those  data  requirements  identified  in 
the  Reliability  Program  Plan  must  also  appear  in  the  CDRL. 

(2)  The  PA  should  review  all  DIDs  and  assure,  through  * 

tailoring,  that  the  preparation  instructions  in  the  DID  are 
compatible  with  task  requirements  as  specified  in  the  SOW. 


MAINTAINABILITY  DATA  ITEM  DESCRIPTIONS  (DID) 


The  following  is  a  list  of  data  item  descriptions  associated  with 
the  maintainability  tasks  specified  in  MIL-STD-470A. 


101 

DI-R-7103 

Maintainability  Program  Plan 

103 

DI-R-7104 

Maintainability  Status  Report 

104 

DI-R-7105 

Data  Collection,  Analysis  and 
Corrective  Action  System,  Reports 

201 

DI-R-7106 

Maintainability  Modeling  Report 

202 

DI-R-7107 

Maintainability  Allocations  Report 

203 

DI-R-7108 

Maintainability  Predictions  Report 

204 

DI-R-7085A 

Failure  Mode,  Effects  and  Criticality 
Analysis  (FMECA)  Report 

205 

DI-R-7109 

Maintainability  Analysis  Report 

206 

DI-R-7110 

Maintainability  Design  Criteria  Plan 

207 

DI-R  7111 

Inputs  to  the  Detailed  Maintenance 
Plan  and  Logistics  Support  Analysis 

301 

DI-R-7112 

Maintainability  Demonstration  Test 
Plan 

C-2 


301  DI-R-2129 


301  DI-R-7113 


Plan,  Maintainability  Demonstration 
(DI  -R-2129  is  to  be  used  only  when 
MIL-STD-471  is  designated  as  the  basis 
for  MIL-STD-470A,  Task  301) 

Report,  Maintainability  Demonstration 
(to  be  used  only  when  MIL-STD-471  and 
its  associated  DI-R-2130A  are  not 
designated  as  a  basis  for 
MIL-STD-470A,  Task  301) 


Notes:  (1)  Only  data  items  specified  in  the  CDRL  are 

deliverable.  Therefore,  those  data  requirements  identified  in 
the  Reliability  Program  Plan  must  also  appear  in  the  CDRL. 

(2)  The  PA  should  review  all  DIDs  and  assure,  through 
tailoring,  that  the  preparation  instructions  in  the  DID  are 
compatible  with  task  requirements  as  specified  in  the  SOW. 


C-3 


The  following  Data  Item  Descriptions  are  listed  for  reference 
purposes  only.  They  have  not  been  linked  to  any  specific  task 
number  in  any  MIL-STD. 


Data  Item 


Description 


DI-RELI-80247 
DI-RELI-80248 
DI-RELI-80249 
DI-RELI  80253 
DI-RELI-80254 
DI-RELI-80261 

DI-RELI-80322 

DI-RELI-80323 

DI-RELI-80373 

DI-RELI-80374 

DI-3549A 

DI-7094 

DI-R-3541/R-109-2 

DI-R-3547/R-115-2 


Thermal  Survey  Report 
Vibration  Survey  Report 
Environmental  Stress  Screening  Report 
Failed  Item  Analysis  Report 
Corrective  Action  Plan 

Production  Inspection  Equipment  Test  Systems 
Engineering  Design  Data 

Quality  Conformance  Inspection  &  Test 
Procedures 

Certification  Demonstration  Procedures 

Equipment  Inspection/Testing  Report 

Failure  ANalysis  Summary  List 

Repair  Level  Analysis  Report  (RLA) 

Reliability  Block  Diagrams  and  Math 
Models  Report 

Computer-Programmed  Math  Model  for 
Reliability 

R&M  Report  on  Commercial  Equipment 


